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ABSTRACT 

It is still under debate whether learners’ interaction data within e-learning and/or open learning environments could be 
considered as reflections of their learning experiences to be effective or not. Therefore, it is meaningful to explore the 
nature of these interactions and to make meaningful conclusions. This study aims to explore what the nature of these 
interactions in an e-learning environment and to describe whether there is a meaningful pattern in their interaction data. 
For this purpose, a course on Computer Networks and Communication was designed in an e-learning platform, where 
learners could receive real-time responses and monitor their process through dashboards as recommendations for their 
learning process. 31 metrics were gathered from database records, which yielded a common factor with six sub -factors, 
where the highest correlation was between learners -learning dashboards interactions and learners-learning objects. In 
addition, this factorial structure could be considered a holistic view of a learning experience based on the interaction 
within an e-leaming environment. Another finding of this study indicated that learners’ interaction with learning 
dashboards had been one and meaningful dimension of their overall learning experiences. The results of this study 
presents instructional design cues and pedagogical outcomes. 
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1. INTRODUCTION 

Learning can be defined as interactive and complex process among learners, instructors and learning 
resources not only in face-to-face environment but also in e-learning environment. For effective learning, 
interaction is an important component of online learning environments in terms of learners, teachers and 
learning context (Anderson, 2003; Arbaugh & Benbunan-Fich, 2007; Joksimovic et al., 2015). It is further 
emphasized that learners’ learning experiences rely heavily on the interactions within these e-learning 
environments (Agudo-Peregrina et al., 2014; Duval, 2011). 

The factorial structure of this interaction during an e-learning task has been explained by different 
interactional engagements, which is generally explained by student-student and student-content interaction 
types (Moore, 1989; Wanstreet, 2006). In addition, based on the pedagogical effectiveness of ICT tools, the 
interaction between learner-interface (Hastings, 2013; Hillman et al., 1994) and learner-system interaction 
(Bouhnik & Marcus, 2006) types have been incorporated into the model to complete our understanding. 
Hence, learners’ interaction with those components within an e-learning environment yield the emergence of 
learning experiences (Parrish, 2009). 

When the research in learning analytics, academic analytics, and informal learning networks are 
reviewed, one could easily observe the difficulties in productive analysis to forecast the possible interactions 
simply by using the interaction types; and, although this interaction is paramount; yet, it is not enough to 
infer whether learning is realized or not (Friensen & Kuskis, 2013; Simonson, 2012). Therefore, it is still 
being explored the nature of the relationship between interaction types and learning outcomes (Joksimovic et 
al., 2015). Furthermore, there is no consensus yet to point out which interaction type is more important to 
choose during running learning analytics (Duval & Verbert, 2012). It remains salient to quest the learning 
experiences themselves (Veletsianos, 2015); furthermore, more research is needed to define learning 
experiences across and/or within related domains. 
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Therefore, the purpose of this study is to model learners’ learning experiences based on their interactions 
in an e-learning environment. The study was designed to understand the nature of interactions and to observe 
whether these interactions display an observable pattern with the following guiding questions: 

RQ1. What is the nature of learners’ interactions in an e-learning environment? 

RQ2. What is the relationship between learners’ interaction types in an e-learning environment? 

RQ3. In this e-learning environment, would learners’ interaction yield a meaningful learning experience 
as a structure? 

The paper proposes a way to use LMS datasets as a factor to predict learning success by analyzing what 
are the major types of interactions among learners and which patterns would be indicative of learning 
experiences. It provides relevant recommendations to instructional designers to encourage specific types of 
interaction during e-learning. We hope that the interaction factor modelled learners’ learning experiences 
would be great to be used in a larger multi-variable predictive model to provide finer -grained predictions of 
learners' academic performances. 


2. METHOD 


2.1 Context 

The context of this study is an e-learning course module designed, developed, and titled Computer Networks 
and Communication. The e-learning environment was Moodle 2.8, where the database was redesigned to 
gather the necessary data for the purpose of this study. The course duration was 12 weeks and the course 
delivery was a hybrid one. Each week, two hours were completed face to face, where students were 
provided guidance and were given the instructions for the following week. 

The expected outcome of the Computer Networks and Communication course was to comprehend the 
foundations of computer networks, to design computer networks, and practice in running and maintaining 
networks. When designing the learning objects for this course, these expected outcomes were taken into 
account. In the e-learning environment, each learning object was designed in accordance with SCORM V.3 
in the form of digital book chapters, course video recordings, educational games, and educational videos. In 
addition, discussion activities and learning tasks were provided through the e-learning environment. 

The e-learning environment was also embedded personalized learning dashboards, which provide 
information to students about their learning process in order to improve their learning performances. These 
learning dashboards were voluntarily available to students and each dashboard displays data calculated 
through a learning analytics process (data extraction, pre-processing, visualization, action, and improvement). 

2.2 Participants 

The participants in this study were 126 undergraduate students attending Computer Networks and 
Communication course in major state universities in Turkey. The mean of pre-test scores on course content 
(ranging from a min 0 to a max 25) was less than one points. The two groups of students interacted within the 
e-learning environment developed by the researchers. 

2.3 Data Sources 

Date and time stamps for each learner activity in the e-learning environment were stored in the system 
database. Performing data processing, 31 additional metrics were defined to be collected as data sources. 
These data were queried through MySql queries to be processed later. The data set was pre-processed and 
combined to define 31 metrics. The metrics were related to certain learning activity and/or behavioral data 
realized during certain learning task. 


254 



13th International Conference on Cognition and Exploratory Learning in Digital Age (CELDA 2016) 


2.4 Data Analysis 

In order to explore the nature of the interactions, as part of feature selection and factoring, principal 
component analysis (PCA) was executed (Kantardzic, 201 1 ). This analysis initiates the process by m times of 
variables in the dataset, runs reduction and rotation analysis, and yield k times linear components (k<m). In 
order to explore the relations between factors, a correlation analysis was run. Then, in order to observe 
whether these learning experiences are hidden within the navigational patterns embedded in the related 
factors, a hierarchical factorial analysis was run. Finally, the corresponding fit indices were reviewed 
(RMSEA, CFI, GFI, NNFI) to check whether the model fitted with the data. 


3. RESULTS 


3.1 What is the Nature of Learners’ Interactions in an E-Learning 
Environment? 

This study is designed to investigate learners’ interactional behaviors in an e-learning environment to infer to 


what extend this experience carries meaning about their learning processes. A total of 31 metrics (variables) 
related to their interaction and behaviors were generated to be analyzed. These 31 variables from their 
behavioral data indicated a correlation with each other; therefore, the rotation in PCA was chosen to be direct 
oblimin rotation, which is preferred when assuming correlations between components (Alpar, 2011; Field. 
2009). PCA was executed over 31 interactional data with 126 observations. Before the analysis, 
Kaiser-Meyer-Olkin (KMO) analysis was checked to see whether the sampling is acceptable and it was 
found that the results were above the acceptable range (KMO=.89) (Field, 2009). Barlett sphericity test also 
indicated that the correlation between items was acceptable for principal factor analysis (y_ 2 (465) =6003.66, 
pc. 001). 

Table 2. The Results of PCA 


Factor Loadings 

Metrics 

Factor 1 Factor 2 Factor 3 Factor 4 Factor 5 Factor 6 

g_Dashboard3view 

.959 

g_ Dashboard2view 

.943 

g_ Dashboard4view 

.937 

g_ Dashboard_view 

.895 

g_ Dashboard2view 

.843 

g_DashboardDuration 

.513 

o_DiscussionTitle 

.624 

o_DiscussionMessage 

.589 

o_DiscussionMessagePoint 

.577 

g_DiscussionNavigation 

.425 

g_DiscussionDuration 

.389 

g_DiscussionReading 

.369 

s_LObj ectDuration 

.997 

g_LObjectView 

.953 
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e_BookView 



.890 




s_BookDuration 



.874 




e_BookOpen 



.850 




e_CourseVideosView 



.530 




g_QuizFeedbackView 



.468 




e_OtherResourcesView 



.467 




g_AssignmentNavigation 



.388 




o_GlossaryCreate 




.970 



g_GlossaryView 




.952 



o_MessagetoInstructor 





.909 


o_ParticipationToChat 





.638 


o_SendingMessage 





.562 


g_MessageNavigation 





.518 


s_QuizDuration 






.768 

o_CompletedQuiz 






.734 

o_FinishedAssignment 






.515 

g_QuizNavigation 






.510 

Eigenvalues 

16.36 

3.15 

2.04 

1.57 

1.31 

1.08 

Explained Variances % 

52.78 

10.18 

6.58 

5.07 

4.22 

3.50 


PCA results indicated there were six factors with an eigen value greater than 1 and the factor loadings 
greater than 0.35. The overall explained variation was found to be 82.35%. These factors and their related 
items are described below: 

• Factor 1 (F-l), consists of 5 items related to learners’ behavioral data related to their interaction with 

dashboards, thus, named as “learner-learning dashboard interaction”. 

• Factor 2 (F-2), consists of six items related to learners’ interaction data in Forum discussions, thus 

named as “learner-learner interaction”. 

• Factor 3 (F-3), consists of seven items related to learners’ interaction data in accessing learning 

objects, and one item related to examination feedback, and one related to navigation between 
learning tasks. Exam feedback was provided as a response to their quizzes, embedded within 
learning tasks. These feedback information is provided with a button interaction, available for 
learners on a voluntarily base. As to the learning tasks, each learning task was provided to 
learners within the course materials and are available to them when more details are sought. 
Therefore, this factor is titled as “learner-learning object interaction”. 

• Factor 4 (F-4), consists of two items related to learners’ interactional data with the glossary; thus, 

named as “learner -glossary interaction”. 

• Factor 5 (F-5), consists of six items related to learners’ interactional data during messaging with 

each other; thus, named as “learner-messaging interaction”. 

• Factor 6 (F-6), consists of three items related to learners’ interactional data with short exams, and 

one item related to their submission task. Since these data are related to their assessment 
experiences, this factor is named as “learner-assessment interaction”. 
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3.2 What is the Relationship between Learners’ Interaction Types in an 
e-Learning Environment? 


The correlation matrix, obtained from the measurement model is presented in Table 2. 

Table 2. The Correlation Matrix 



F-l 

F-2 

F-3 

F-4 

F-5 

F-6 

F-l 

1.000 






F-2 

.211 

1.000 





F-3 

.580 

.163 

1.000 




F-4 

.375 

.178 

.441 

1.000 



F-5 

.251 

.195 

.347 

.273 

1.000 


F-6 

.389 

.066 

.297 

.063 

.079 

1.000 


When the correlation matrix is considered, the highest correlation was between factor 1 and 3; and, factor 
3 and 4; the lowest, on the other hand, was between factor 2 and 6, and factor 4 and 6. These findings 
indicate that learner-learning object interaction has a positive and medium level correlation with 
learner-learning dashboards and learner-glossary interaction. Furthermore, learner-assessment interaction has 
a positive yet low correlation with learner-learner interaction and learner-glossary interaction. 

PCA results indicated six different factors available when understanding the learning experiences in this 
particular context. This result is an expected outcome in an e-learning environment when considering 
learners navigate among the learning sources, initiate and continue with mutual messaging among peers, and 
engage in learning related activities. 


3.3 Would Learners’ Interaction Yield a Meaningful Learning Experience as a 
Structure? 

In order to observe whether the existing six factors would yield an upper construct, a hierarchical factor 
analysis was run. The analysis results are presented in Figure 1 . 
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Chi-Square=13.33, df=9, p-value=0.14833, RMSEA=0.062 

Figure 1. T values in hierarchical factor analysis 
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In order to see the model -data fit in the structural model, the fit and error indices are presented in Table 3. 

Table 3. Fit Indices 


Fit indices 

Acceptable values 

Analysis results 

CFI 

CFI > 0.90 

0.97 

GFI 

GFI > 0.90 

0.97 

NNFI 

NNFI > 0.90 

0.96 

RMSEA 

RMSEA < 0.08 

0.062 


When the structure in Figure 1 and the values in Table 3 are evaluated, it can be concluded that the fit 
indices are within the acceptable range and the model-fit indices are established. 

These findings indicate that when learners’ interactions in an e-learning environment are examined, it can 
be concluded that their behavioral patterns indicate that they develop a learning experience within this 
context, consisting of six different components (i.e., “learner-learning dashboard interaction”, 
“learner-learner interaction”, “learner-learning object interaction”, “learner-glossary interaction”, “learner- 
messaging interaction”, and “learner-assessment interaction”). Each of these sub-components of the overall 
learning experience produces valuable data for understanding the e-learning process from learners’ 
behavioral data. 


4. CONCLUSION 

This study revealed the learning experience as a construct with six sub-factors. In addition to the existing 
literature, where the two interactions were heavily reported (learner -learner and learner-content), this study 
extends the nature of relations to learner-learning dashboard interaction”, “learner-glossary interaction”, 
“learner-messaging interaction”, and “learner-assessment interaction”, which represent the nature of 
interaction. Factor reduction analyses also yielded plausible data to help us predict learners’ interaction 
during their e-learning sessions. 

Hierarchical factor analysis yielded these six sub-factors could be an indicator of an upper construct. This 
finding supports the theoretical framework in that learners’ experience is shaped through connections in 
social context; moreover, they take charge of their learning process (Macfadyen & Dawson, 2010; Vygotsky, 
1978). In addition, this finding also supports the existing assumptions in learning analytics research findings 
in that learners’ interactions within a learning environment represent their learning experiences (Bousbia & 
Belamri, 2014; Dyckhoff et al., 2012; Tempelaar et al., 2015). 

This study also found that learners’ interaction with learning dashboards is a sub-component of their 
overall learning. This finding has various insights for learning analytics researchers. Learning dashboards 
enable learners to monitor their own learning experiences; therefore, when designing instruction, emphasis 
should be placed on designing and developing interactional opportunities with learning dashboards. 

When the relationship among the sub-factors of a learning experience is examined, the highest correlation 
was found to be between learner-learning object and learner-learning dashboards. This relationship might be 
an indicator of a tendency toward using learning dashboards, if students are in interaction with learning 
objects. Although existing literature reports that learner-content interaction is the highest predictor of success 
(Bernard et al., 2009), there are some contradictory findings in predicting success (Joksimovic et al., 2015; 
Agudo-Peregrina et al., 2014). Furthermore, in the literature, researchers have reported that learners spend 
most of their time in interacting with content (Macfadyen & Dawson, 2010). This study also supported this 
finding in that learners had spent significant time interacting with learning objects compared to others. 
Therefore, regardless of their achievement, we can speculate that as learners interact more with learning 
objects, they tend to use learning dashboards accordingly. On the other hand, it can also be argued that as 
they spend more time with learning objects, they get engaged with personal activities; thus, leading to lessen 
their interaction time with their peers (Dennen, 2013). 
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The overall purpose of this study is to model learners’ learning experiences based on their interactions in 
an e-learning environment and to propose design ideas as well as pedagogical cues for online course 
instructors. The emerged interactional patterns could be a source when designing e-learning course materials 
(Pardo, 2014; Pistilli et al., 2014). Furthermore, the relationship between learning experiences and outcomes 
could be further explored when designing personalized learning environments (Greller & Drachsler, 2012; 
Siemens, 2013; Spector, 2013). To conclude, these interactional patterns could be explored in various 
contexts with several learner characteristics considering individual differences of learners. 
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